Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[vstest] Only collect stdout of orchagent_restart_check in vstest #2597

Merged
merged 2 commits into from
Jan 4, 2023
Merged

[vstest] Only collect stdout of orchagent_restart_check in vstest #2597

merged 2 commits into from
Jan 4, 2023

Conversation

bingwang-ms
Copy link
Contributor

What I did
This PR is to fix the flaky test cases in vstest.
We frequently saw test failures in PR testing like below

def test_warm_restart_all_interfaces(self,dvs,testlog):
        self.setup_db(dvs)
    
        tbl = swsscommon.Table(self.cdb,"PORT")
        for key in tbl.getKeys():
            self.add_storm_control_on_interface(dvs,key,"broadcast",1000000)
            self.add_storm_control_on_interface(dvs,key,"unknown-unicast",2000000)
            self.add_storm_control_on_interface(dvs,key,"unknown-multicast",3000000)
        #dvs.runcmd("config save -y")
        # enable warm restart
        dvs.warm_restart_swss("true")
    
        # freeze orchagent for warm restart
        (exitcode, result) = dvs.runcmd("/usr/bin/orchagent_restart_check")
>       assert result == "RESTARTCHECK succeeded\n"
E       AssertionError: assert 'RESTARTCHECK...unction 503\n' == 'RESTARTCHECK succeeded\n'
E         + RESTARTCHECK succeeded
E         - RESTARTCHECK failed
E         - profiling:/__w/1/s/common/.libs/libswsscommon_la-countertable.gcda:Merge mismatch for function 503

The output is produced by gcov. However, it's collected into the output of orchagent_restart_check and leads to test failure because the output is not as expected.

- profiling:/__w/1/s/common/.libs/libswsscommon_la-countertable.gcda:Merge mismatch for function 503

This PR fixes the issue by adding an argument include_stderr to runcmd. When include_stderr is set to False, only stdout is collected.

Why I did it
This PR is to fix the flaky test cases in vstest.

How I verified it
Verified by running vstest locally.

Details if related

@bingwang-ms bingwang-ms marked this pull request as ready for review January 4, 2023 00:54
@bingwang-ms bingwang-ms requested a review from prsunny as a code owner January 4, 2023 00:54
prsunny
prsunny previously approved these changes Jan 4, 2023
View reviewed changes
@bingwang-ms bingwang-ms changed the title Only collect stdout of orchagent_restart_check in vstest [vstest] Only collect stdout of orchagent_restart_check in vstest Jan 4, 2023
@bingwang-ms bingwang-ms merged commit f496ab3 into sonic-net:master Jan 4, 2023
yxieca pushed a commit that referenced this pull request Jan 4, 2023
@bingwang-ms @yxieca
[vstest] Only collect stdout of orchagent_restart_check in vstest (#2597
bbccc68 
)

* Only collect stdout of orchagent_restart_check in vstest
@Junchao-Mellanox Junchao-Mellanox mentioned this pull request Jan 30, 2023
Closed
dprital added a commit to dprital/sonic-buildimage that referenced this pull request Jan 30, 2023
@dprital
[submodule] Advance sonic-swss pointer
78f8d99 
Update sonic-swss submodule pointer to include the following:
* a2a483d [acl] Add new ACL key BTH_OPCODE and AETH_SYNDROME  ([sonic-net#2617](sonic-net/sonic-swss#2617))
* 9d1f66b [bfdorch] add local discriminator to state DB ([sonic-net#2629](sonic-net/sonic-swss#2629))
* c54b3d1 Vxlan tunnel endpoint custom monitoring APPL DB table. ([sonic-net#2589](sonic-net/sonic-swss#2589))
* 7f03db2 Fix potential risks ([sonic-net#2516](sonic-net/sonic-swss#2516))
* 383ee68 [refactor]Refactoring sai handle status ([sonic-net#2621](sonic-net/sonic-swss#2621))
* cd95972 Fix issue 13341 ARP entry can be out of sync between kernel and APPL_DB if multiple updates are received from RTNL ([sonic-net#2619](sonic-net/sonic-swss#2619))
* a01470f Remove TODO comments that are no longer relevant ([sonic-net#2622](sonic-net/sonic-swss#2622))
* d058390 Changed the BFD default detect multiplier to 10x ([sonic-net#2614](sonic-net/sonic-swss#2614))
* d78b528 [MuxOrch] Enabling neighbor when adding in active state ([sonic-net#2601](sonic-net/sonic-swss#2601))
* 4ebdad1 [routesync] Fix for stale dynamic neighbor ([sonic-net#2553](sonic-net/sonic-swss#2553))
* 8857f92 Added new attributes for Vnet and Vxlan ecmp configurations. ([sonic-net#2584](sonic-net/sonic-swss#2584))
* b6bbc3e Revert [voq][chassis]Add show fabric counters port/queue commands (2522) ([sonic-net#2611](sonic-net/sonic-swss#2611))
* 52406e2 Add missing parameter to on_switch_shutdown_request method. ([sonic-net#2567](sonic-net/sonic-swss#2567))
* 4ac9ad9 Increase diff coverage to 80% ([sonic-net#2599](sonic-net/sonic-swss#2599))
* 8a0bb36 Handle Mac address 'none' ([sonic-net#2593](sonic-net/sonic-swss#2593))
* f496ab3 [vstest] Only collect stdout of orchagent_restart_check in vstest ([sonic-net#2597](sonic-net/sonic-swss#2597))
* 1dab495 Avoid aborting orchagent when setting TUNNEL attributes ([sonic-net#2591](sonic-net/sonic-swss#2591))
* 4395cea Fix neighbor doesn't update all attribute ([sonic-net#2577](sonic-net/sonic-swss#2577))

Signed-off-by: dprital <drorp@nvidia.com>
@dprital dprital mentioned this pull request Jan 30, 2023
Merged
8 tasks
liat-grozovik pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Feb 1, 2023
@dprital
[submodule] Advance sonic-swss pointer (#13551)
4c3aafc 
Update sonic-swss submodule pointer to include the following:
* a2a483d [acl] Add new ACL key BTH_OPCODE and AETH_SYNDROME  ([#2617](sonic-net/sonic-swss#2617))
* 9d1f66b [bfdorch] add local discriminator to state DB ([#2629](sonic-net/sonic-swss#2629))
* c54b3d1 Vxlan tunnel endpoint custom monitoring APPL DB table. ([#2589](sonic-net/sonic-swss#2589))
* 7f03db2 Fix potential risks ([#2516](sonic-net/sonic-swss#2516))
* 383ee68 [refactor]Refactoring sai handle status ([#2621](sonic-net/sonic-swss#2621))
* cd95972 Fix issue 13341 ARP entry can be out of sync between kernel and APPL_DB if multiple updates are received from RTNL ([#2619](sonic-net/sonic-swss#2619))
* a01470f Remove TODO comments that are no longer relevant ([#2622](sonic-net/sonic-swss#2622))
* d058390 Changed the BFD default detect multiplier to 10x ([#2614](sonic-net/sonic-swss#2614))
* d78b528 [MuxOrch] Enabling neighbor when adding in active state ([#2601](sonic-net/sonic-swss#2601))
* 4ebdad1 [routesync] Fix for stale dynamic neighbor ([#2553](sonic-net/sonic-swss#2553))
* 8857f92 Added new attributes for Vnet and Vxlan ecmp configurations. ([#2584](sonic-net/sonic-swss#2584))
* b6bbc3e Revert [voq][chassis]Add show fabric counters port/queue commands (2522) ([#2611](sonic-net/sonic-swss#2611))
* 52406e2 Add missing parameter to on_switch_shutdown_request method. ([#2567](sonic-net/sonic-swss#2567))
* 4ac9ad9 Increase diff coverage to 80% ([#2599](sonic-net/sonic-swss#2599))
* 8a0bb36 Handle Mac address 'none' ([#2593](sonic-net/sonic-swss#2593))
* f496ab3 [vstest] Only collect stdout of orchagent_restart_check in vstest ([#2597](sonic-net/sonic-swss#2597))
* 1dab495 Avoid aborting orchagent when setting TUNNEL attributes ([#2591](sonic-net/sonic-swss#2591))
* 4395cea Fix neighbor doesn't update all attribute ([#2577](sonic-net/sonic-swss#2577))

Signed-off-by: dprital <drorp@nvidia.com>
StormLiangMS pushed a commit that referenced this pull request Feb 10, 2023
@bingwang-ms @StormLiangMS
[vstest] Only collect stdout of orchagent_restart_check in vstest (#2597
13b5adf 
)

* Only collect stdout of orchagent_restart_check in vstest
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lolyu lolyu lolyu approved these changes

@prsunny prsunny prsunny left review comments

Assignees
No one assigned
Labels
Included in 202205 Branch Included in 202211 Branch Request for 202205 Branch Request for 202211 Branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@bingwang-ms @prsunny @lolyu @yxieca @StormLiangMS